approximating martingale process
Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space
Ride-hailing services, such as Uber, Lyft, and Didi Chuxing, have become a popular stochastic process problem being studied in operations research. Approximating the optimal policy of matching drivers and customers in realtime is especially difficult due to the large state space and the combinatorial nature of the problem. In [1], the authors consider a Markov decision process (MDP) model of a ride-hailing service system and innovatively decompose the MDP actions by sequentially assigning tasks to available drivers due to the large action space. Then, the reinforcement learning algorithm proximal policy optimization (PPO) [2] is adopted for the ride-hailing system's control policy optimization. On the other hand, Multiclass Queueing Networks (MQNs) are a special class of stochastic processing networks, a classic problem in operations research. In order to find the optimal control policy of such a network, [3] formulated the MQN problem with Poisson arrival and exponential service time as an MDP, also using reinforcement learning algorithm PPO from [2] to optimize the network's policy.